Method and apparatus for determining the attentional focus of individuals within a group

ABSTRACT

A method, apparatus and computer program product are provided to effectively determine the attentional focus of each of a plurality of individuals, such as the members of a group. In the context of a method, information is received identifying an attentional focus of each of a plurality of individuals. The method also includes determining, with a processor, a relationship between a threshold and a number of individuals with an attentional focus having a predefined association with a location. Further, the method includes causing an action to be performed based upon the relationship.

TECHNOLOGICAL FIELD

An example embodiment of the present invention relates generally to thedetermination of the attentional focus of individuals within a groupand, more particularly, to the performance of an action based at leastin part upon the attentional focus of individuals within a group.

BACKGROUND

Individuals may focus their attention upon various objects. Further, theattentional focus of an individual may change over time, sometimesrelatively frequently, as the individual initially focuses theirattention on a first object and then subsequently refocuses theirattention on a second object. Particularly in a group, it may bedifficult to determine the attentional focus of members of the group,either individually or in the aggregate.

It may sometimes be desirable to determine the attentional focus of theindividuals of a group. For example, it may be useful to determine thatevery member of a group was focusing their attention upon a camera inadvance of capturing an image of the group. As another example, in aninstance in which a plurality of guards are monitoring a particularobject or a predefined location, it may be useful to determine that eachof guards or at least a certain number or percentage of the guards hadtheir attentional focus upon the particular object or location to ensurethat the appropriate security was being provided. As yet anotherexample, it may be useful to understand the attentional focus of theindividuals within a group when confronted with a plurality ofadvertisements or other marketing materials in order to gauge theeffectiveness of the advertisements and other marketing materials.However, the difficulties associated with determining the attentionalfocus of an individual within a group may make it difficult to providesuch information.

BRIEF SUMMARY

A method, apparatus and computer program product are provided inaccordance with an example embodiment of the present invention in orderto effectively determine the attentional focus of each of a plurality ofindividuals, such as the members of a group. As such, the method,apparatus and computer program product of an example embodiment maycause an action to be performed at least partially based upon theattentional focus of each of the plurality of individuals. Consequently,the action which is therefore caused to be performed may produceimproved or more effective results as a result of the consideration ofthe attentional focus of each of the plurality of individuals.

In one embodiment, a method is provided that includes receivinginformation identifying an attentional focus of each of a plurality ofindividuals, such as by receiving head position information for each ofthe plurality of individuals. For example, the method may receiveinformation identifying an attentional focus by receiving informationfrom a wearable device worn by each of the plurality of individuals. Themethod also includes determining, with a processor, a relationshipbetween a threshold and a number of individuals with an attentionalfocus having a predefined association with a location. Further, themethod includes causing an action to be performed based upon therelationship.

The method of one embodiment determines a relationship by determining arelationship between a minimum threshold and the number of individualswith an attentional focus that includes the location. In thisembodiment, the method may cause an action to be performed by causingthe action to be performed in response to the number of individuals withan attentional focus that includes the location being at least as greatas the minimum threshold. In this embodiment, the action that is causedto be performed may be the capture of an image of the location or thecapture of at least some of the plurality of individuals. The method ofanother embodiment determines a relationship by determining arelationship between a maximum threshold and the number of individualswith an attentional focus that fails to include the location. In thisembodiment, the method may cause an action to be performed by causingthe action to be performed in response to the number of individuals withan attentional focus that fails to include the location being at leastas great as the maximum threshold. In this embodiment, the action thatis caused to be performed may be an action that is performed to draw theattentional focus of at least some of the individuals to the location.

In another embodiment, an apparatus is provided that includes at leastone processor and at least one memory including computer program codewith the at least one memory and the computer program code configuredto, with the processor, cause the apparatus to at least receiveinformation identifying an attentional focus of each of a plurality ofindividuals. The at least one memory and the computer program code arealso configured to, with the processor, cause the apparatus to determinea relationship between a threshold and a number of individuals with anattentional focus having a predefined association with a location.Further, the at least one memory and the computer program code areconfigured to, with the processor, cause the apparatus to cause anaction to be performed based upon the relationship.

The at least one memory and the computer program code may be configuredto, with the processor, cause the apparatus of one embodiment todetermine a relationship by determining a relationship between a minimumthreshold and the number of individuals with an attentional focus thatincludes the location. In this embodiment, the at least one memory andthe computer program code may be configured to, with the processor,cause the apparatus to cause an action to be performed by causing theaction to be performed in response to the number of individuals with anattentional focus that includes the location being at least as great asthe minimum threshold. In this embodiment, the action that is caused tobe performed may be the capture of an image of the location or thecapture of at least some of the plurality of individuals. The at leastone memory and the computer program code may be configured to, with theprocessor, cause the apparatus of another embodiment to determine arelationship by determining a relationship between a maximum thresholdand the number of individuals with an attentional focus that fails toinclude the location. In this embodiment, the at least one memory andthe computer program code may be configured to, with the processor,cause the apparatus to cause an action to be performed by causing theaction to be performed in response to the number of individuals with anattentional focus that fails to include the location being at least asgreat as the maximum threshold. In this embodiment, the action that iscaused to be performed may be an action that is performed to draw theattentional focus of at least some of the individuals to the location.

In a further embodiment, a computer program product is provided thatincludes at least one non-transitory computer-readable storage mediumhaving computer-executable program code portions stored therein with thecomputer-executable program code portions including program codeinstructions for receiving information identifying an attentional focusof each of a plurality of individuals. The computer-executable programcode portions also include program code instructions for determining arelationship between a threshold and a number of individuals with anattentional focus having a predefined association with a location.Further, the computer-executable program code portions include programcode instructions for causing an action to be performed based upon therelationship.

The program code instructions for determining a relationship inaccordance with one embodiment may include program code instructions fordetermining a relationship between a minimum threshold and the number ofindividuals with an attentional focus that includes the location. Inthis embodiment, the program code instructions for causing an action tobe performed may include program code instructions for causing theaction to be performed in response to the number of individuals with anattentional focus that includes the location being at least as great asthe minimum threshold. In this embodiment, the action that is caused tobe performed may be the capture of an image of the location or thecapture of at least some of the plurality of individuals. The programcode instructions for determining a relationship in accordance withanother embodiment may include program code instructions for determininga relationship between a maximum threshold and the number of individualswith an attentional focus that fails to include the location. In thisembodiment, the program code instructions for causing an action to beperformed may include program code instructions for causing the actionto be performed in response to the number of individuals with anattentional focus that fails to include the location being at least asgreat as the maximum threshold. In this embodiment, the action that iscaused to be performed may be an action that is performed to draw theattentional focus of at least some of the individuals to the location.

In yet another embodiment, an apparatus is provided that includes meansfor receiving information identifying an attentional focus of each of aplurality of individuals. The apparatus also includes means fordetermining a relationship between a threshold and a number ofindividuals with an attentional focus having a predefined associationwith a location. Further, the apparatus includes means for causing anaction to be performed based upon the relationship.

BRIEF DESCRIPTION OF THE DRAWINGS

Having thus described certain embodiments of the present invention ingeneral terms, reference will now be made to the accompanying drawings,which are not necessarily drawn to scale, and wherein:

FIG. 1 is an illustration of a plurality of individuals, each exhibitingan emotional response and having an attentional focus;

FIG. 2 is an illustration of a plurality of individuals, some of whichhave a different attentional focus.

FIG. 3 is a block diagram of an apparatus that may be specificallyconfigured in accordance with an example embodiment of the presentinvention;

FIG. 4 is a flowchart illustrating the operations performed, such as bythe apparatus of FIG. 3, in order to cause an action to be performedbased upon the attentional focus of at least some individuals inaccordance with an example embodiment of the present invention;

FIG. 5 is a flowchart illustrating the operations performed, such as bythe apparatus in FIG. 3, in order to cause an action to be performed inresponse to at least a threshold number of individuals having anattentional focus that includes a location in accordance with an exampleembodiment of the present invention;

FIG. 6 is a flowchart illustrating the operations performed, such as bythe apparatus in FIG. 3, in order to cause an action to be performed inresponse to at least a threshold number of individuals failing to payattention to a location in accordance with an example embodiment of thepresent invention; and

FIG. 7 is a flowchart illustrating the operations performed, such as bythe apparatus in FIG. 3, in order to cause an action to be performedbased at least in part upon the emotional response of at least some of aplurality of individuals in accordance with an example embodiment of thepresent invention.

DETAILED DESCRIPTION

Some embodiments of the present invention will now be described morefully hereinafter with reference to the accompanying drawings, in whichsome, but not all, embodiments of the invention are shown. Indeed,various embodiments of the invention may be embodied in many differentforms and should not be construed as limited to the embodiments setforth herein; rather, these embodiments are provided so that thisdisclosure will satisfy applicable legal requirements. Like referencenumerals refer to like elements throughout. As used herein, the terms“data,” “content,” “information,” and similar terms may be usedinterchangeably to refer to data capable of being transmitted, receivedand/or stored in accordance with embodiments of the present invention.Thus, use of any such terms should not be taken to limit the spirit andscope of embodiments of the present invention.

Additionally, as used herein, the term ‘circuitry’ refers to (a)hardware-only circuit implementations (e.g., implementations in analogcircuitry and/or digital circuitry); (b) combinations of circuits andcomputer program product(s) comprising software and/or firmwareinstructions stored on one or more computer readable memories that worktogether to cause an apparatus to perform one or more functionsdescribed herein; and (c) circuits, such as, for example, amicroprocessor(s) or a portion of a microprocessor(s), that requiresoftware or firmware for operation even if the software or firmware isnot physically present. This definition of ‘circuitry’ applies to alluses of this term herein, including in any claims. As a further example,as used herein, the term ‘circuitry’ also includes an implementationcomprising one or more processors and/or portion(s) thereof andaccompanying software and/or firmware. As another example, the term‘circuitry’ as used herein also includes, for example, a basebandintegrated circuit or applications processor integrated circuit for amobile phone or a similar integrated circuit in a server, a cellularnetwork device, other network device, and/or other computing device.

As defined herein, a “computer-readable storage medium,” which refers toa non-transitory physical storage medium (e.g., volatile or non-volatilememory device), can be differentiated from a “computer-readabletransmission medium,” which refers to an electromagnetic signal.

Referring now to FIG. 1, a plurality of individuals are depicted. Theindividuals may be members of a group, such as a group of familymembers, a group of business acquaintances, a tour group, etc.Alternatively, the individuals may be unaffiliated with one another, butmay be located proximate to one another, such as by being within thesame room, by being within a predefined distance of one another or thelike. Regardless, the individuals are generally proximate one another asshown in FIG. 1. The individuals may be located in a variety of settingsand may be engaged in various activities. For example, the individualsmay be visitors in a museum, gallery or exhibit, may be attendees at aperformance, such as a concert, may be customers at a restaurant, may besecurity guards policing a particular object or location, or the like.

Each individual may have an attentional focus and an emotional response.The attentional focus of an individual is the object, location or thelike upon which the individual is focusing their attention. Theattentional focus of an individual may be determined in various mannersincluding being determined based upon the position of the individual'shead and, more particularly, based upon the direction in which theindividual is gazing. The emotional response of an individual is theoutward or public manifestation of the emotional state of an individual,such as represented by a smile, laughter, furrowing of the eyebrows,wrinkling of the nose or the like.

With respect to attentional focus, FIG. 2 depicts a scenario in whichfour individuals are located proximate one another, such as within aroom of an art gallery. As shown by the dashed lines, two individualshave the same attentional focus upon painting 14 as a result of theirlooking at the painting. As also shown by the dashed lines, the otherindividuals have their attentional focus upon different locations withone individual having their attentional focus upon painting 16 andanother individual having their attentional focus upon painting 18.

In the illustrated embodiment, each of the plurality of individuals iswearing a wearable device and, more particularly, ahead-worn device,such as a head mounted display, headphones or ear buds, that may includeone or more sensors for providing information regarding the attentionalfocus or emotional response of the individual. While each of theforegoing examples may be capable of providing output, the wearabledevice of other embodiments need not provide output. Additionally, whilethe foregoing examples are of head-worn devices, the wearable device maybe worn by other portions of an individual's body, such as a pendant.For the purposes of illustration, but not of limitation, the method,apparatus and computer program product of an example embodiment will bedescribed below in conjunction with a head mounted device, although themethod, apparatus and computer program product may be employed inconjunction with other types of wearable devices in other embodiments.In the illustrated embodiment, the head mounted displays 10 are in theform of a pair of glasses. However, the head mounted display may beembodied in other forms including a head mounted visor or a helmetmanner display. For example, the head mounted display may be in a formof a helmet worn by a motorcyclist, a pilot or they like. Regardless ofthe manner in which the head mounted display is embodied, the headmounted display permits a user to view their surroundings, while alsopermitting information to be presented to the user, thereby augmentingthe user's view of their surroundings. The information presented by thehead mounted display may augment the objects in the scene viewed throughthe head mounted display, such as by identifying or otherwise providingmore information regarding one or more of the objects viewed through thehead mounted display. Alternatively, the information presented by thehead mounted display may be unrelated to the objects in the scene viewedthrough the head mounted display, but may otherwise provide informationthat may be of interest to the user. Regardless of the type ofinformation presented by the head mounted display, a head mounteddisplay as exemplified by the glasses may support augmented reality andother applications.

As shown in FIG. 3, an apparatus 20 that may be specifically configuredin accordance with an example embodiment of the present invention isillustrated. The apparatus may be embodied in one or more of the headmounted displays 10 worn by the individuals. Alternatively, theapparatus may be embodied by a computing device that is remote from thehead mounted display, but that is in communication therewith, such asvia wireless communication, e.g., via Bluetooth communication, Wi-Fi oranother wireless network, or via wired communication. For example, acomputing device, such as a personal digital assistant (PDA), mobiletelephone, smartphone, pager, mobile television, gaming device, laptopcomputer, camera, tablet computer, touch surface, video recorder,audio/video player, radio, electronic book, positioning device (e.g.,global positioning system (GPS) device), or any combination of theaforementioned, and other types of voice and text communicationssystems, may embody the apparatus of one embodiment. Regardless of themanner in which the apparatus is embodied, the head mounted display ofeach individual may be configured to communicate information identifyingthe individual's attentional focus to another device, such as to theother head mounted displays worn by other individuals in an instance inwhich the apparatus is distributed between the plurality of head mounteddisplays or to an external computing device that may embody theapparatus in another embodiment.

Regardless of the manner in which the apparatus 20 is embodied, theapparatus may include or otherwise be in communication with a processor22, a memory device 24 and a communication interface 26. It should benoted that while FIG. 3 illustrates one example of a configuration of anapparatus for receiving and responding to user input, numerous otherconfigurations may also be used to implement embodiments of the presentinvention. As such, in some embodiments, although devices or elementsare shown as being in communication with each other, hereinafter suchdevices or elements should be considered to be capable of being embodiedwithin the same device or element and thus, devices or elements shown incommunication should be understood to alternatively be portions of thesame device or element.

In some embodiments, the processor 22 (and/or co-processors or any otherprocessing circuitry assisting or otherwise associated with theprocessor) may be in communication with the memory device 24 via a busfor passing information among components of the apparatus. The memorydevice may include, for example, one or more volatile and/ornon-volatile memories. In other words, for example, the memory devicemay be an electronic storage device (e.g., a computer readable storagemedium) comprising gates configured to store data (e.g., bits) that maybe retrievable by a machine (e.g., a computing device like theprocessor). The memory device may be configured to store information,data, content, applications, instructions, or the like for enabling theapparatus 20 to carry out various functions in accordance with anexample embodiment of the present invention. For example, the memorydevice could be configured to buffer input data for processing by theprocessor. Additionally or alternatively, the memory device could beconfigured to store instructions for execution by the processor.

As noted above, the apparatus 20 may be embodied by a computing device,such as a head mounted display 10 or a computing device in communicationwith the head mounted display, configured to employ an exampleembodiment of the present invention. However, in some embodiments, theapparatus may be embodied as a chip or chip set. In other words, theapparatus may comprise one or more physical packages (e.g., chips)including materials, components and/or wires on a structural assembly(e.g., a baseboard). The structural assembly may provide physicalstrength, conservation of size, and/or limitation of electricalinteraction for component circuitry included thereon. The apparatus maytherefore, in some cases, be configured to implement an embodiment ofthe present invention on a single chip or as a single “system on achip.” As such, in some cases, a chip or chipset may constitute meansfor performing one or more operations for providing the functionalitiesdescribed herein.

The processor 22 may be embodied in a number of different ways. Forexample, the processor may be embodied as one or more of varioushardware processing means such as a coprocessor, a microprocessor, acontroller, a digital signal processor (DSP), a processing element withor without an accompanying DSP, or various other processing circuitryincluding integrated circuits such as, for example, an ASIC (applicationspecific integrated circuit), an FPGA (field programmable gate array), amicrocontroller unit (MCU), a hardware accelerator, a special-purposecomputer chip, or the like. As such, in some embodiments, the processormay include one or more processing cores configured to performindependently. A multi-core processor may enable multiprocessing withina single physical package. Additionally or alternatively, the processormay include one or more processors configured in tandem via the bus toenable independent execution of instructions, pipelining and/ormultithreading.

In an example embodiment, the processor 22 may be configured to executeinstructions stored in the memory device 24 or otherwise accessible tothe processor. Alternatively or additionally, the processor may beconfigured to execute hard coded functionality. As such, whetherconfigured by hardware or software methods, or by a combination thereof,the processor may represent an entity (e.g., physically embodied incircuitry) capable of performing operations according to an embodimentof the present invention while configured accordingly. Thus, forexample, when the processor is embodied as an ASIC, FPGA or the like,the processor may be specifically configured hardware for conducting theoperations described herein. Alternatively, as another example, when theprocessor is embodied as an executor of software instructions, theinstructions may specifically configure the processor to perform thealgorithms and/or operations described herein when the instructions areexecuted. However, in some cases, the processor may be a processor of aspecific device (e.g., a head mounted display) configured to employ anembodiment of the present invention by further configuration of theprocessor by instructions for performing the algorithms and/oroperations described herein. The processor may include, among otherthings, a clock, an arithmetic logic unit (ALU) and logic gatesconfigured to support operation of the processor.

Meanwhile, the communication interface 26 may be any means such as adevice or circuitry embodied in either hardware or a combination ofhardware and software that is configured to receive and/or transmit datafrom/to a head mounted display 10, such as a computing device embodiedby the head mounted display, in an embodiment in which the apparatus 20is remote from the head mounted display. Additionally, the communicationinterface may be configured to communicate with other components of thecomputing device in an instance in which the apparatus is embodied by acomputing device embodied by the head mounted display or with a remotecomputing device in an instance in which the apparatus is separate fromthe head mounted display.

In this regard, the communication interface 26 may include, for example,an antenna (or multiple antennas) and supporting hardware and/orsoftware for enabling communications wirelessly. Additionally oralternatively, the communication interface may include the circuitry forinteracting with the antenna(s) to cause transmission of signals via theantenna(s) or to handle receipt of signals received via the antenna(s).For example, the communications interface may be configured tocommunicate wirelessly with the head mounted displays 10, such as viaWi-Fi, Bluetooth or other wireless communications techniques. In someinstances, the communication interface may alternatively or also supportwired communication. As such, for example, the communication interfacemay include a communication modem and/or other hardware/software forsupporting communication via cable, digital subscriber line (DSL),universal serial bus (USB) or other mechanisms. For example, thecommunication interface may be configured to communicate via wiredcommunication with other components of the computing device including,for example, other components of the head mounted display in anembodiment in which the apparatus 20 is embodied by the head mounteddisplay.

Referring now to FIG. 4, the operations performed, such as by theapparatus 20 of FIG. 3, in order to cause an action to be performedbased at least in part upon the attentional focus of at least some ofthe individuals are depicted. In this regard, depending upon theattentional focus of some of the individuals, different actions may becaused to be performed, thereby tailoring the action to the attentionalfocus of the individuals such that the resulting action may be moreeffective.

As shown in FIG. 3 and with reference to block 30 of FIG. 4, theapparatus 20 may receive information identifying the attentional focusof each of a plurality of individuals. The apparatus may thereforeinclude means, such as the processor 22, communication interface 26 orthe like, for receiving the information identifying the attentionalfocus of each of the plurality of individuals. The attentional focus ofthe individuals may be determined in various manners and, as such, theapparatus, such as the processor, the communication interface or thelike, may receive the information identifying the attentional focus invarious manners and from various devices.

In one example, the information identifying the attentional focus of anindividual may be head position information. Consequently, the positionof an individual's head may be tracked. For example, a digital compassmay be carried by the user's head, such as by including a digitalcompass within the head mounted display 10 worn by the individual, withthe digital compass configured to provide the apparatus with headposition information. In one embodiment, the location of the individualmay also be determined, such as by a global positioning system (GPS).For example, a head mounted display may also include a GPS for providingthe location information. As such, the apparatus 20, such as theprocessor 22, communication interface 26 or the like, of one embodimentmay receive the information identifying the attentional focus of anindividual in the form of head position information as well as, in oneembodiment, information identifying the location of the individual, suchas from a head mounted display worn by the individual. Based upon theposition of the individual's head and, in some embodiments, also basedupon the location of the individual, the apparatus, such as theprocessor, may determine the attentional focus of the individual. Asanother example, a camera may be carried by the individual, such as byincluding the camera within the head mounted display. The camera of thisembodiment may be configured to capture images that include one or morepredefined features in the environment, such as one or moreceiling-mounted visual markers, that permit the position and orientationof the individual's head and, therefore, the individual's attentionalfocus to be tracked.

Additionally or alternatively, the gaze of an individual may be tracked,such as with a gaze tracker carried, for example, by the head mounteddisplay 10 and configured to monitor the movement and behavior of theindividual's eyes. In this embodiment, the apparatus 20, such as theprocessor 22, the communication interface 26 or the like, may thereforereceive information identifying the direction in which the individual isgazing from a respective gaze tracker. Thus, the attentional focus ofthe individual may be determined by the apparatus, such as theprocessor, based upon the gaze of the individual.

Still further, one or more cameras, such as camera 28 of FIG. 2, maycapture images of the plurality of individuals. As such, the apparatus,such as the processor, communication interface or the like, may receiveinformation, such as images, from one or more cameras such that thedirection and orientation of the individuals' heads and bodies may bereferenced in order to determine the direction in which the individualsare looking, thereby defining the attentional focus of the individuals.In one embodiment, one or more environmental cameras, such as camera 28of FIG. 2, monitor everyone in a room. The images captured by theenvironmental cameras may be provided to a central computing devicewhich processes the images and correlates between each camera to obtainan overall picture of where individuals are looking, thereby identifyingthe attentional focus of the individuals. Or, each individual may haveone or more wearable cameras which may, for example, be included in thehead-mounted display. In this embodiment, the head-mounted cameras maytrack the gaze or head direction of the people around them. Thehead-mounted cameras may provide the images to a central computingdevice for a determination of the attentional focus of each individual.As described by some of the foregoing examples, the informationidentifying the attentional focus of an individual may be provided by anexternal device, that is, a device that is physically separate from theindividual whose attentional focus is identified. For example, a camera,such as camera 28 of FIG. 2, is external to the individuals, but mayprovide information identifying the attentional focus of an individual.

In each of the foregoing embodiments, the attentional focus may bedetermined, such as by the head mounted display 10, prior to theprovision of information to the apparatus 20 such that the informationthat is received by the apparatus explicitly identifies the attentionalfocus of each of the individuals. Alternatively, the informationreceived by the apparatus identifying the attentional focus of eachindividual may be the more granular information that was collected, suchas information identifying the position of the individual's head, thelocation of the individual, the gaze of the individual or the like, suchthat the apparatus, such as the processor 22, may thereafter determinethe attentional focus of each individual based upon the information thatwas received.

The apparatus 20 of one embodiment, such as the processor 22, maydetermine the attentional focus of an individual even in an instance inwhich the individual is fully or partially obscured from a view fromapparatus 20. Indeed, the apparatus of this embodiment may provide for adistributed attentional focus determination as distinguished, forexample, from a centralized gaze tracking solution in which theapparatus relies upon information regarding the attentional focus of theindividuals that is obtained from a single camera or a single gazetracker and which is therefore limited to the determination of theindividuals that can be perceived. However, an apparatus that providesfor a distributed attentional focus determination may receiveinformation regarding the attentional focus of an individual fromanother device, such as a head mounted display of the individual, suchthat the attentional focus of the individual may be determined even ifthe individual is fully or partially obscured from the view of theapparatus.

Referring now to block 32 of FIG. 4, the apparatus 20 may be configuredto determine a relationship between a threshold, such as a minimumthreshold or a maximum threshold, and the number of individuals with anattentional focus having a predefined association with a location. Thus,the apparatus may include means, such as the processor 22 or the like,for determining the relationship between the threshold and the number ofindividuals with an attentional focus having a predefined associationwith the location. The predefined association of the attentional focuswith the location may be defined in various manners. In one embodiment,for example, the attentional focus may have a predefined associationwith the location in an instance in which the attentional focus includesthe location. In another embodiment, the attentional focus may have apredefined association with the location in an instance in which theattentional focus fails to include the location, that is, in an instancein which the attentional focus is directed somewhere different than thelocation. In a further embodiment, the attentional focus may have apredefined association with the location in an instance in which theduration of the attentional focus upon the location satisfies athreshold, such as by having at least a predefined duration, therebyindicating that the individual is paying substantial attention to thelocation. In yet another embodiment, the attentional focus may have apredefined association with the location in an instance in which theattentional focus is proximate the location, such as within a predefinedangular or distance range of the location, even though the individual mynot be staring directly at the location.

The location that is considered in determining whether the attentionalfocus of an individual has a predefined association therewith may bedetermined in various manners. For example, the location may bepredetermined, such as the location of an object that is to be securedby a plurality of guards, the location of a particular performer uponstage, the location of a camera or the like. However, the location neednot be predetermined and may, instead, be determined based upon thelocation to which all or at least a certain number or percentage of theindividuals either direct their attentional focus or fail to directtheir attentional focus, as explained below.

Referring now to block 34 of FIG. 4, the apparatus 20 may also cause anaction to be performed based upon the relationship. Thus, the apparatusmay include means, such as the processor 22, the communication interface26 or the like, for causing an action to be performed based upon therelationship, such as by transmitting an instruction to another deviceto cause the action to be performed as shown in FIG. 3. In oneembodiment, the relationship may be expressed in terms of the number ofindividuals with an attentional focus having a predefined associationwith a location exceeding a threshold, equaling the threshold or beingless than the threshold. The threshold may be defined in various mannersincluding a number of individuals, a percentage of individuals or thelike. As such, the apparatus of one embodiment may cause an action to beperformed in an instance in which the number of individuals with anattentional focus having a predefined association with a location is atleast as great as the threshold, while in an another embodiment, theapparatus may be configured to cause the action to be performed in aninstance in which the number of individuals with an attentional focushaving a predefined association with a location is less than thethreshold.

By way of an example and with reference to FIG. 5, the apparatus 20,such as the processor 22, communication interface 26 or the like, mayreceive information identifying an attentional focus of each of aplurality of individuals as described above in conjunction with block 30of FIG. 4. See block 40. In this embodiment, the apparatus, such as theprocessor or the like, also determines a relationship between a minimumthreshold and the number of individuals with an attentional focus thatincludes a location. See block 42 of FIG. 5. The minimum threshold maybe defined in various manners including a number of individuals, apercentage of individuals or the like. In this embodiment, theapparatus, such as a processor, the communication interface or the like,may then cause the action to be performed in response to the number ofindividuals with an attentional focus that includes the location beingat least as great as the minimum threshold. See block 44. The actionthat is caused to be performed may vary depending upon the situation. Inone embodiment, however, an image of the location may be captured, suchas by a head-worn or otherwise body-worn camera or a camera 28 locatedelsewhere in the room, in response to determining that the number ofindividuals with an attentional focus that includes the location is atleast as great as the minimum threshold since the common attentionalfocus may indicate that something interesting is occurring at thelocation. Additionally or alternatively, an image of at least some ofthe plurality of individuals may be captured. With reference to FIG. 1,for example, a camera 12 may capture an image of the individuals oncethe apparatus determines that a least a threshold number of theindividuals have an attentional focus that includes the camera, therebyinsuring that at least some and, in some instances, all of theindividuals are facing the camera.

By way of another embodiment, reference is now made to FIG. 6 in whichthe apparatus 20, such as the processor 22, communication interface 26or the like, is again configured to receive information identifying anattentional focus of each of a plurality of individuals. See block 50.In this embodiment, the apparatus, such as the processor, is configuredto determine a relationship between a maximum threshold and the numberof individuals with an attentional focus that fails to include thelocation, such as represented by individuals having an attentional focuson a different location. See block 52. As such, in block 54 of FIG. 6,the apparatus, such as the processor, the communication interface or thelike, may also be configured to cause the action to be performed inresponse to the number of individuals with an attentional focus thatfails to include the location being at least as great as the maximumthreshold. In this embodiment, the action is therefore caused to beperformed in an instance in which it is determined that a number ofindividuals are not paying attention to the location. In one exampleembodiment, in the apparatus, such as the processor, the communicationinterface or the like, may cause an action to be performed that servesto draw the attentional focus of at least some of the individuals to thelocation, such as by shining a brighter light upon the location,advertising or promoting the location, causing an announcement to beissued to one or more of the individuals regarding the location,highlighting the location via an overlay or other indicia, such as avisual attractant, presented by the individual's head mounted display orthe like. In a more detailed example in which a plurality of guards areto be securing an object at a particular location, the action that iscaused to be performed in response to determining that fewer than athreshold number of the guards have the particular location within theirattentional focus may be the issuance of a notification, such as anemail, a text message, an automated voice recording, a personal visitfrom their supervisor, the presentation of indicia, such as a visualattractant, by the head mounted display of the guards or the like,advising the guards to pay additional attention to the object that isbeing secured. Although the notification may be issued to all of theindividuals, the apparatus, such as the processor, of another embodimentmay cause the notification to only be issued to certain individuals,such as the individuals closest to the location, with the otherindividuals being excluded from receipt of the notification. In additionto or instead of causing a notification to be issued, an image of theindividuals, such as the guards, may be captured to evidence the failureto pay attention to a particular location and/or an image of thelocation that does serve as the attentional focus of the individuals soas to determine what distracted the individuals from the location thatwas intended to be their attentional focus.

As described above, the apparatus 20 may cause various actions to beperformed in accordance with embodiments of the present invention. Inone embodiment, the action that is performed may be the provision ofinformation regarding the attentional focus of the plurality ofindividuals. As a more particular example, the attentional focus of aplurality of audience members may be determined and informationregarding the attentional focus of the audience members may be provided,such as in the form of real time or near real time feedback. Based uponthe feedback, one or more aspects of the performance may be altered,such as by shining a brighter light upon a particular performer who isfailing to attract the attentional focus of a sufficient number ofaudience members or by de-emphasizing another performer, such as byreducing the lighting on the other performer, who is garnering anexcessive amount of attention from the audience members. In thisembodiment, the apparatus may utilize different thresholds in thedetermination as to whether an action is to be performed with thethreshold being varied based upon various parameters, such as proximityto the location, object to be viewed at the location, etc.

In another example embodiment, the disbursement of guests within amuseum, gallery or other exhibit may be monitored and announcements orother suggestions may be provided by the management of the museum,gallery or other exhibit in an instance in which an excessive number ofindividuals have their attentional focus upon a particular work, therebyresulting in the overcrowding of the region surrounding the work, or inan instance in which an unexpectedly small number of individuals areviewing another work. As the foregoing examples demonstrate, the methodand apparatus may cause many of a wide variety of actions to beperformed in response to a determination of the relationship between thethreshold and a number of individuals with an attentional focus having apredefined association with a location.

In addition to or instead of considering the attentional focus of eachof a plurality of individuals, the method, apparatus and computerprogram product of an example embodiment may consider the emotionalresponse of each of a plurality of individuals, such as by consideringthe outward emotional response representative of the current emotionalstate of each of a plurality of individuals. As shown in FIG. 7, theoperations performed by an apparatus 20, such as shown in FIG. 3, inaccordance with an example embodiment in which various actions arecaused to be performed at least partially in response to the emotionalresponse of each of a plurality of individuals are illustrated. In thisregard, the apparatus is configured to receive information identifyingan emotional response of each of a plurality of individuals. See block60. Thus, the apparatus may include means, such as the processor 22, thecommunication interface 26 or the like, for receiving informationidentifying the emotional response of each of a plurality ofindividuals.

The emotional response of an individual may be determined in variousmanners and, as such, the apparatus 20 may receive the informationidentifying the emotional response in a variety of manners from variousdevices. By way of example, a capacitive sensor may track changes in theshape of the user's face, thereby permitting various facial expressions,such as a smile or a grimace, to be identified. The capacitive sensormay be proximate the user's face and, in one embodiment, is carried byor included within the head mounted display 10 worn by the individual.Additionally or alternatively, a microphone and an associated audioprocessor may be configured to detect audible responses, such aslaughter or a groan, and, in some embodiments, to perform voice analysisupon the audible responses. In this embodiment, a head mounted displaymay include the microphone and, in some embodiments, the associatedaudio processor. Further, a physiological sensor may monitorphysiological signals which may, in turn, be representative of anemotional response of an individual. For example, anelectroencephalographic (EEG) sensor may detect brain waves which, inturn, may be indicative of the emotional response of an individual. Inthis embodiment, the EEG sensor(s) may be incorporated within the headmounted display worn by the individual. Additionally or alternatively, asensor may be carried by a head mounted display or more generally by ahat or by eyeglasses with the sensor being responsive to a change in theposition of the individual's eyebrows. As such, the sensor of thisembodiment may provide information indicative of the individual'seyebrows being raised or being furrowed. Still further, an image of theindividual could be captured and image processing techniques may beemployed to determine if the individual is smiling, grimacing orotherwise displaying a particular emotional response. While any one ofthe sensors may be used individually to provide information indicativeof an individual's emotional response, combinations of the sensors maybe employed in order to determine even more complex emotion responses ofthe user.

Regardless of the manner in which the information identifying theemotional response of an individual is collected, the apparatus 20, suchas the processor 22, the communication interface 26 or the like, isconfigured to receive the information identifying the emotionalresponse. In this regard, the information identifying the emotionalresponse that is received by the apparatus may be information collected,such as by a sensor, from the individual that may, in turn, be analyzedby the apparatus, such as a processor, in order to determine theemotional response of the individual. Alternatively, the emotionalresponse of the individual may be determined based upon the informationthat has been collected prior to the provision of the information to theapparatus such that the information that is subsequently provided to theapparatus more explicitly defines the emotion response of theindividual. In yet another embodiment, the information identifying theemotional response that is received by the apparatus may be informationidentifying the emotional state of an individual, such as in adistributed embodiment as described below in which the individual maynot be within view of the apparatus.

The apparatus 20 of one embodiment, such as the processor 22, maydetermine the emotional response of an individual even in an instance inwhich the individual is fully or partially obscured from a view fromapparatus 20. Indeed, the apparatus of this embodiment may provide for adistributed emotional response determination as distinguished, forexample, from a centralized solution in which the apparatus relies uponinformation regarding the emotional response of the individuals that isobtained from a single camera or a single sensor and which is thereforelimited to the determination of the individuals that can be perceived.However, an apparatus that provides for a distributed emotional responsedetermination may receive information regarding the emotional responseof an individual from another device, such as a head mounted display ofthe individual, such that the emotional response of the individual maybe determined even if the individual is fully or partially obscured fromthe view of the apparatus.

Referring now to block 62 of FIG. 7, the apparatus 20 may also beconfigured to determine a relationship between a threshold, such as amaximum threshold or a minimum threshold, and a number of individualswith an emotional response that satisfies a predefined condition. Thus,the apparatus may include means, such as the processor 22 or the like,for determining the relationship between a threshold and a number ofindividuals with an emotional response that satisfies the predefinedcondition. The predefined condition may be defined in various mannersand, in one embodiment, is a predefined emotional state or a range orfamily of related emotional states. For example, the predefinedcondition may be the emotional state of being happy with individualshaving an emotional response of smiling or laughing being considered tosatisfy the predefined condition, while users who have an emotionalresponse of grimacing fail to satisfy the predefined condition. Asanother example, a predefined condition may relate to a predefined rateof change of the emotional state of an individual such that individualssatisfy this predefined condition in an instance in which theiremotional responses change at a rate that satisfies the predefined rateof change. Alternatively the predefined condition may be defined to be amanifestation of an emotional state that is shared by each of theindividuals such that the emotional response of the individual satisfiesthis predefined condition only in an instance in which each of theindividuals has the same emotional response. In this embodiment, themanifestation of an emotional state may be considered to be shared byeach of the individuals in an instance in which the manifestation of theemotional state of each of the individuals is the same or substantiallythe same, such as by being within a predefined range of one another.

As shown in block 64 of FIG. 7, the apparatus 20 is also configured tocause an action to be performed based upon the relationship between thethreshold and the number of individuals with an emotional response thatsatisfies the predefined condition. Thus, the apparatus of thisembodiment also includes means, such as the processor 22, thecommunication interface 26 or the like, for causing an action to beperformed. The threshold may be defined in various manners including anumber of individuals, a percentage of individuals or the like. Therelationship between the threshold and the number of individuals with anemotional response that satisfies the predefined condition may bedefined in various manners including the number of individuals with anemotional response that satisfies the predefined condition exceeding thethreshold, falling below the threshold or equaling the threshold.

In one embodiment, the apparatus 20, such as the processor 22, maydetermine the relationship between the threshold and the number ofindividuals with an emotional response that satisfies the predefinedcondition by determining whether the number of individuals with apredefined emotional response is at least as great as the threshold. Inan instance in which the number of individuals with a predefinedemotional response is at least as great as the threshold, the apparatus,such as the processor, the communication interface 26 or the like, maycause an action to be performed.

Various actions may be performed based upon the relationship between thethreshold and the number of individuals with an emotional response thatsatisfies the predefined condition. In the foregoing example in whichthe number of individuals with a predefined emotional response is atleast as great as the threshold, the action that is caused to beperformed may be the capturing of an image of the individuals. In thisregard, the predefined emotional response may be a smile such that ifthe number of individuals who smile is at least as great as thethreshold, the apparatus 20 may cause an image of the individuals to becaptured. The resulting image may be further improved by not onlyrequiring the number of individuals with the predefined emotionalresponse, e.g., a smile, to be at least as great as the threshold, butby also requiring the number of individuals who have an attentionalfocus upon the location in which the camera resides to be at least asgreat as the threshold as described above. In this embodiment, theresulting image may therefore be captured only after determining that atleast the threshold number, e.g., all, of the individuals are smilingand facing the camera.

In another example in which the predefined condition is laugher, theapparatus 10, such as the processor 22, may determine whether the numberof individuals who are laughing are at least as great as a threshold. Inan instance in which the number of individuals who are laughing are atleast as great as the threshold, the apparatus 20, such as the processor22, the communication interface 26 or the like, may direct that an imageof the individuals be captured. For example, the individuals may be thepatrons of a restaurant who are seated at a table and an image of thepatrons may be captured while at least a threshold number of the patronsare laughing such that the image may thereafter be utilized formarketing or advertising purposes. While described above with thepredefined condition being laughter, the foregoing example is alsoapplicable with other predefined conditions. Instead of or in additionto capturing an image of the individuals, the action that is caused tobe performed may be the capturing of an image of the location that isthe subject of the attentional focus of the individuals once theemotional responses of the individuals have satisfied the predefinedcondition, such as laughter, fright, terror, shock, revulsion, etc.,thereby capturing an image of the object that may have caused theemotional response. As yet another example, the action that is caused tobe performed may be the recordation or the provision of informationassociated with satisfaction of the threshold, such as the time and dateat which the threshold was surpassed and/or information identifying atleast some of the individuals, e.g., the location of the individuals atthe time that the threshold was surpassed, etc.

In another embodiment, the apparatus 20, such as a processor 22, maydetermine the relationship between the threshold and the number ofindividuals with an emotional response that satisfies the predefinedcondition by determining whether the number of individuals with asubstantially similar emotional response is at least as great as thethreshold. In this embodiment, the apparatus, such a processor,communication interface 26 or the like, may then cause an action to beperformed in response to the number of individuals with thesignificantly similar emotional response being at least as great as thethreshold. Although various actions may be caused to be performed, theaction to be performed in one embodiment may be the provision ofinformation identifying the emotional responses of at least some of theindividuals, thereby providing feedback as to a relatively commonemotional response, such as to an advertisement, a movie, a performance,a speech or the like. Alternatively, the action to be performed in aninstance in which at least a threshold number of individuals expresssignificantly similar emotional responses may be to capture an image ofat least some of the individuals.

In this embodiment, a significantly similar emotional response may bedefined in one embodiment to be those emotional responses that arerepresentative of the same emotional state, such as the emotionalresponses of laughter and smiling being representative of the sameemotional state of being happy. Additionally or alternatively, theemotional responses may be assigned a value in accordance with apredefined scale that extends from very unhappy to neutral to veryhappy. In this embodiment, the significantly similar emotion responsesmay be defined to be those emotional responses that fall within apredefined range or window of one another upon the scale of emotionalresponses.

In another embodiment, the information identifying the emotionalresponses that is received by the apparatus 20, such as the processor22, communication interface 26 or the like, may include informationregarding a rate of change of an emotional state of each of theplurality of individuals. Alternatively, the information identifying theemotional responses that is received by the apparatus may be providedover the course of time such that the apparatus, such as the processor,may determine the rate of change of an emotional state of an individualfrom the changes in the emotional response of the individual over thecourse of time. In this embodiment, the predefined condition may be apredefined rate of change such that the apparatus, e.g., the processoror the like, may determine the number of individuals having an emotionalresponse that experiences at least the predefined rate of change andmay, thereafter, determine the relationship between the threshold andthe number of individuals with an emotional response that satisfies thepredefined rate of change. The apparatus, such as the processor,communication interface or the like, may then cause an action to beperformed based upon the resulting relationship.

As described above, FIGS. 4-7 illustrate flowcharts of an apparatus,method, and computer program product according to example embodiments ofthe invention. It will be understood that each block of the flowcharts,and combinations of blocks in the flowcharts, may be implemented byvarious means, such as hardware, firmware, processor, circuitry, and/orother devices associated with execution of software including one ormore computer program instructions. For example, one or more of theprocedures described above may be embodied by computer programinstructions. In this regard, the computer program instructions whichembody the procedures described above may be stored by a memory device34 of an apparatus 20 employing an embodiment of the present inventionand executed by a processor 32 of the apparatus. As will be appreciated,any such computer program instructions may be loaded onto a computer orother programmable apparatus (e.g., hardware) to produce a machine, suchthat the resulting computer or other programmable apparatus implementsthe functions specified in the flowchart blocks. These computer programinstructions may also be stored in a computer-readable memory that maydirect a computer or other programmable apparatus to function in aparticular manner, such that the instructions stored in thecomputer-readable memory produce an article of manufacture the executionof which implements the function specified in the flowchart blocks. Thecomputer program instructions may also be loaded onto a computer orother programmable apparatus to cause a series of operations to beperformed on the computer or other programmable apparatus to produce acomputer-implemented process such that the instructions which execute onthe computer or other programmable apparatus provide operations forimplementing the functions specified in the flowchart blocks.

Accordingly, blocks of the flowcharts support combinations of means forperforming the specified functions and combinations of operations forperforming the specified functions for performing the specifiedfunctions. It will also be understood that one or more blocks of theflowcharts, and combinations of blocks in the flowcharts, can beimplemented by special purpose hardware-based computer systems whichperform the specified functions, or combinations of special purposehardware and computer instructions.

Many modifications and other embodiments of the inventions set forthherein will come to mind to one skilled in the art to which theseinventions pertain having the benefit of the teachings presented in theforegoing descriptions and the associated drawings. Therefore, it is tobe understood that the inventions are not to be limited to the specificembodiments disclosed and that modifications and other embodiments areintended to be included within the scope of the appended claims.Moreover, although the foregoing descriptions and the associateddrawings describe example embodiments in the context of certain examplecombinations of elements and/or functions, it should be appreciated thatdifferent combinations of elements and/or functions may be provided byalternative embodiments without departing from the scope of the appendedclaims. In this regard, for example, different combinations of elementsand/or functions than those explicitly described above are alsocontemplated as may be set forth in some of the appended claims.Although specific terms are employed herein, they are used in a genericand descriptive sense only and not for purposes of limitation.

That which is claimed:
 1. A method comprising: receiving information identifying an attentional focus of each of a plurality of individuals; determining, with a processor, a relationship between a threshold and a number of individuals with an attentional focus having a predefined association with a location; and causing an action to be performed based upon the relationship.
 2. A method according to claim 1 wherein determining a relationship comprises determining a relationship between a minimum threshold and the number of individuals with an attentional focus that includes the location, and wherein causing an action comprises causing the action to be performed in response to the number of individuals with an attentional focus that includes the location being at least as great as the minimum threshold.
 3. A method according to claim 2 wherein causing the action further comprises causing an image of the location to be captured.
 4. A method according to claim 2 wherein causing the action further comprises causing an image of at least some of the plurality of individuals to be captured.
 5. A method according to claim 1 wherein determining a relationship comprises determining a relationship between a maximum threshold and the number of individuals with an attentional focus that fails to include the location, and wherein causing an action comprises causing the action to be performed in response to the number of individuals with an attentional focus that fails to include the location being at least as great as the maximum threshold.
 6. A method according to claim 5 wherein causing the action further comprises causing the action to be performed to draw the attentional focus of at least some of the individuals to the location.
 7. A method according to claim 1 wherein receiving information identifying an attentional focus comprises receiving head position information for each of the plurality of individuals.
 8. A method according to claim 1 wherein receiving information identifying an attentional focus comprises receiving information from a wearable device worn by each of the plurality of individuals.
 9. An apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the processor, cause the apparatus to at least: receive information identifying an attentional focus of each of a plurality of individuals; determine a relationship between a threshold and a number of individuals with an attentional focus having a predefined association with a location; and cause an action to be performed based upon the relationship.
 10. An apparatus according to claim 9 wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to determine a relationship by determining a relationship between a minimum threshold and the number of individuals with an attentional focus that includes the location, and wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to cause an action by causing the action to be performed in response to the number of individuals with an attentional focus that includes the location being at least as great as the minimum threshold.
 11. An apparatus according to claim 20 wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to cause the action by causing an image of the location to be captured.
 12. An apparatus according to claim 10 wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to cause the action by causing an image of at least some of the plurality of individuals to be captured.
 13. An apparatus according to claim 10 wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to determine a relationship by determining a relationship between a maximum threshold and the number of individuals with an attentional focus that fails to include the location, and wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to cause an action by causing the action to be performed in response to the number of individuals with an attentional focus that fails to include the location being at least as great as the maximum threshold.
 14. An apparatus according to claim 13 wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to cause the action by causing the action to be performed to draw the attentional focus of at least some of the individuals to the location.
 15. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code portions stored therein, the computer-executable program code portions comprising program code instructions for: receiving information identifying an attentional focus of each of a plurality of individuals; determining a relationship between a threshold and a number of individuals with an attentional focus having a predefined association with a location; and causing an action to be performed based upon the relationship.
 16. A computer program product according to claim 15 wherein determining a relationship comprises determining a relationship between a minimum threshold and the number of individuals with an attentional focus that includes the location, and wherein causing an action comprises causing the action to be performed in response to the number of individuals with an attentional focus that includes the location being at least as great as the minimum threshold.
 17. A computer program product according to claim 16 wherein the program code instructions for causing the action further comprise program code instructions for causing an image of the location to be captured.
 18. A computer program product according to claim 16 wherein the program code instructions for causing the action further comprise program code instructions for causing an image of at least some of the plurality of individuals to be captured.
 19. A computer program product according to claim 15 wherein the program code instructions for determining a relationship comprise program code instructions for determining a relationship between a maximum threshold and the number of individuals with an attentional focus that fails to include the location, and wherein the program code instructions for causing an action comprise program code instructions for causing the action to be performed in response to the number of individuals with an attentional focus that fails to include the location being at least as great as the maximum threshold.
 20. A computer program product according to claim 19 wherein the program code instructions for causing the action further comprise program code instructions for causing the action to be performed to draw the attentional focus of at least some of the individuals to the location. 